Method and apparatus for image processing with color constancy

ABSTRACT

Fractal decompression and the Retinex algorithm are combined to produce a color constancy method. By using this approach, color constancy and image compression can be achieved simultaneously.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not Applicable

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

Not Applicable

NOTICE OF MATERIAL SUBJECT TO COPYRIGHT PROTECTION

A portion of the material in this patent document is subject tocopyright protection under the copyright laws of the United States andof other countries. The owner of the copyright rights has no objectionto the facsimile reproduction by anyone of the patent document or thepatent disclosure, as it appears in the United States Patent andTrademark Office publicly available file or records, but otherwisereserves all copyright rights whatsoever. The copyright owner does nothereby waive any of its rights to have this patent document maintainedin secrecy, including without limitation its rights pursuant to 37C.F.R. §1.14.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention pertains generally to image compression anddecompression, and more particularly to fractal image compression anddecompression, and most particularly to obtaining color constancy duringthe decompression of fractally compressed images.

2. Description of Related Art

Data compression may generally be defined as processes of transforminginformation from one representation to another, smaller representationfrom which the original data, or a close approximation thereto, can berecovered by the complementary processes of data decompression. Thecompression and decompression processes are often referred to as codingand decoding. Techniques for compressing digital image data includeMPEG, MJPEG, JPEG, DCT, PNG, wavelet, and fractal.

The storage and transmission of large amounts of data are oftenfacilitated by the use of compression and decompression techniques. Inparticular, the transmission and storage of visual images involves largeamounts of data, and benefits greatly from image compression anddecompression techniques. However, when compressed image data isdecompressed to output on a display device or printer or other outputdevice, problems can result. One problem is to ensure color constancyamong the images produced on different displays or other output devices.

Scene lighting conditions cause two major problems or limitations incolor images compared to the direct human observation of the scenes.First, there is a comparative loss of detail and color in shadow zonesof images captured by both photographic and electronic cameras. This isthe dynamic range problem. Second, changes in the spectral distributionof the illumination source cause color distortions in the images. Thisis the color constancy problem.

Electronic cameras (e.g., based on CCD detector arrays) can acquireimage data across a wide dynamic range. This range is typically wideenough to handle most illumination variations within scenes, and cameraadjustments can usually handle illumination variations from scene toscene. However, this range is usually lost when the image is digitizedor when the image is output to a printer or display, which has a muchmore limited dynamic range.

The color constancy problem typically arises from the spectraldifferences between daylight and artificial lighting. Different filmand/or filters can be used to try to compensate, but these do notprovide any dynamic range compression, and cause detail and color in theshadows to be lost or severely attenuated compared to what a humanobserver would actually see.

The development of image processing systems has been at the heart of therecent digital image revolution. These systems process captured digitalimages to enhance their clarity and details using sophisticated imageprocessing algorithms, resulting in images that are substantially moredetailed and accurate than previously. However, a substantial differenceremains between an image perceived by a person and an image captured andreproduced on a display. Despite improvements in digital imageprocessing systems, they still cannot reproduce images with the samelevel of detail, color constancy, and lightness, as a human perceives.This is due in part because the human eye has a greater dynamic rangecompression than current digital image systems. Dynamic rangecompression refers to the ability to distinguish varying levels oflight. The human eye has a dynamic range compression of about 1000:1,which means that the eye can distinguish about 1000 levels of lightvariation. By contrast, digital image systems typically use 8bits/pixel, which allows for a dynamic range compression of only 256:1.

Current technologies for creating, storing, and display of electronicimages must be duplicated for the various end displays, due todifferences in resolution, dynamic range, and color gamut. This iscurrently overcome by choosing from a set of images on server equipment,or by resizing the image when displayed. Usually, a completely separateimage must be processed to do anything about the dynamic range at all.

The rendering format for a device consists of the height and width of arendered image, and it's color gamut or “dynamic range”. Differentformats are appropriate to different rendering devices, such as a largedynamic range and color gamut for a CRT, a large dynamic range andheight and width for an HDTV device, and a very large height and width,but smaller dynamic range and color gamut for printed media.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to an image processing method andapparatus that produces an output having a single format (e.g.,rendering format, which consists of the height and width of the renderedimage, and it's color gamut or “dynamic range”) that can be usedanywhere (e.g., on any display or other output device). It combines thewell-known properties of fractal compression to resize images with thedynamic range manipulation of retinex. According to the invention,retinex processing is combined with the decoding of fractally compressedimages. The images are compressed using any typical fractal encodingalgorithm, but with the compression in the color directions set for aslower convergence decode. The decode algorithm is replaced by one inwhich range blocks are averaged down and then combined with the domainblock by averaging in, using the range block values and difference asone part of the average, and the existing domain block as the other. Thecompression and decompression algorithms are performed on the logarithmof the image, taking the anti-log at the end of the decompression torender the image again. When this algorithm is iterated, the effect isthat it converges in space to the image, and slowly converges on thecolor as with the retinex algorithm, and therefore can be used to createimages of desired dynamic range. Since fractal decompression can be usedto create images of desired size, this algorithm gives complete controlof the images to match the resolution and dynamic range of any displaydevice or printer.

An aspect of the invention is a method for obtaining an image andprocessing the image by a combination of retinex processing and fractalcompression and decompression techniques. The method also includesencoding the image to produce a compressed image; and decoding thecompressed image to produce a decompressed image. The method furtherincludes compressing the image by a compression algorithm which combinesretinex processing with fractal compression to produce a compressedimage; and decompressing the compressed image by a decompressionalgorithm which combines retinex processing with fractal decompressionto produce a decompressed image of a desired dynamic range and size.

Another aspect of the invention is an apparatus having a processor withan associated memory, including program instructions, which whenexecuted by the processor, cause the processor to process an image by acombination of retinex processing and fractal compression anddecompression techniques, and data used to process the image. Theapparatus also includes an encoder programmed to compress an image by acombined retinex and fractal compression algorithm; and a decoderprogrammed to decompress the compressed image by a combined retinex andfractal decompression algorithm. An image processor includes means forobtaining an image; means for processing the image by a combination ofretinex processing and fractal compression and decompression techniques;and means for outputting the processed image.

A still further aspect of the invention is a machine readable mediumcontaining instructions, which when executed by a machine, cause themachine to perform operations comprising processing an image by acombination of retinex processing and fractal compression anddecompression techniques; and outputting the processed image.

Further aspects of the invention will be brought out in the followingportions of the specification, wherein the detailed description is forthe purpose of fully disclosing preferred embodiments of the inventionwithout placing limitations thereon.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The invention will be more fully understood by reference to thefollowing drawings which are for illustrative purposes only:

FIG. 1A is a flowchart of a retinex/fractal image compression algorithmof the present invention.

FIG. 1B is a flowchart of a retinex/fractal image decompressionalgorithm of the present invention.

FIGS. 2A and 2B are block diagrams of apparatus for carrying out thepresent invention.

FIGS. 3A-B through FIG. 8A-B are images illustrating the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

Referring more specifically to the drawings, for illustrative purposesthe present invention is embodied in the apparatus, flowcharts, andimages generally shown in FIGS. 1A-B through FIGS. 8A-B. It will beappreciated that the apparatus may vary as to configuration and as todetails of the parts, and that the method may vary as to the specificsteps and sequence, without departing from the basic concepts asdisclosed herein.

Color is inherently 3-dimensional. One approach to the present inventionis to handle the color as three planes, and associated with each planeis a third dimension of color intensity. Another is to handle the threecolor dimensions simultaneously, in which case the intensity dimensionbecomes a 3-dimensional vector. These dimensions are herein referred toas “the intensity”, which means the color directions, regardless of howthey are handled.

Introduction

Fractal decompression and the Retinex algorithm are combined in theinvention to produce a new color constancy method. By using thisapproach, color constancy and image compression can be achievedsimultaneously. While the juxtaposition of color constancy, a topicwhich involves negating the effects of lighting and changes in dynamicrange, with fractals, a topic usually identified with dynamic systems orimage compression, may seem unlikely to one skilled in the art, thepresent invention shows that this is not the case. According to theinvention, a fractal decompression algorithm is easily converted into analgorithm for color constancy, by combining it with the retinexalgorithm, and even such a discontinuous mapping can effect a change indynamic range that makes sense to the eye.

The invention has two main components, which are individually known, butare combined in a unique way to produce unexpected results. The Retinexalgorithm is a method for approaching color constancy. There are twopublished types; the invention uses the algorithm known as McCann'99.The invention also uses a fractal compression and decompressionalgorithm. The two algorithms are put together to produce the presentinvention.

The invention uses fractal compression techniques. Fractal compressionreplaces the original image with a function. This function consists ofcontracting the image, usually by 2:1 in each spatial dimension and byp:1 for p>1 in the color dimensions (done separately for each colorplane). Because the method does not use pixels, but rather image blocksas its basic unit, these blocks can be of any of a fairly largevariation of sizes, allowing the image to be zoomed or reduced. Thatmuch has been done before in the prior art, e.g. by Iterated Systems,Inc.

The new method of the present invention allows control over the colordimensions. First of all, the function used to average each image blockwith its target is modified. Then the number p is reduced to be closerto one. In fact p can be chosen equal to one, but this works slowly, soa slightly larger number is chosen. This assures that the components ofthe function converge at a more suitable rate (spatially faster thancolor domain-wise). The averaging step assures that the color domainfunction contains discontinuous diffusion. Consequently, the imageconverges to an image that can be different in size, and different inthe application of color. Since the method gives complete control overthe image display, it can be used to show the same content on differentdisplays or other output devices.

This invention might also be used to provide touch-up and nonlinearediting for images. It is also possible to use it to correct for badlighting or noisy transmission of images by contracting the image andencoding it, then decoding to get rid of noise, while adjustment of theconvergence rate corrects for the lighting.

Possible variations include the following:

(a) Changes in the rates of convergence to produce differentphotographic results.

(b) Use of different decode algorithms, specifically “pixel chaining,”to produce different diffusion results.

(c) Changes in the implementation of the compression strategy thatexplicitly forecast this decompression method.

(d) Use of the variability of size and color rendering to compare imagesin a database of images.

The Retinex Algorithm

The Retinex algorithm, developed originally by Land and McCann, is analgorithm for calculating the proper lightness of each pixel in animage. The principle on which it is based is that parts of the image,due to accommodation, are seen as certain colors based on their contrastwith nearby image elements. Consequently, there is no fixed color thatis perceived as green, only a fixed relationship to the surroundingcolors. Retinex algorithms are described in B. Funt, F. Ciurea, and J.McCann, “Retinex in Matlab,” Journal of Electronic Imaging 13, pp.48-57, January, 2004, which is herein incorporated by reference.

The algorithm is run on the logarithm of the image, which models thechange in image strength on the surface of the retina. To model thelateral connections between cells in the retinal ganglia, a random pathstarting farther away from a given pixel is sampled, and the differencesbetween the pixels in the path are summed. Whenever this sum gets higherthan a maximum threshold, it is replaced by the threshold value.

Consequently, image pixels are calculated as distances down from thisvalue in luminance. This modeling gives an accommodation of decreasinginfluence as the path gets further from the pixel being calculated. Thisalgorithm, which is close to the original Retinex, is called theFrankle-McCann algorithm. If this is repeated for each pixel in theimage, it is not difficult to see that information about the surround ofeach pixel is incorporated in its choice of pixel value.

Another way to effect the Retinex algorithm, dubbed McCann'99, takeseach pixel, and averages it with its immediate neighbor minus theoriginal difference between the two pixels.

Once again, if the result surpasses a pre-ordained maximum, it is resetto the maximum. The choice of which neighbor to average is variedrandomly as the algorithm is iterated. The image is averaged down insuccession until it is a minimum size, and this average is replaced withthe maximum. The algorithm proceeds for a fixed number of iterations,then the result is interpolated up to the next size, and the algorithmrepeats. This version of the algorithm is a diffusion modified by anon-linear reset operation at each level of iteration.

It has been shown that either algorithm, if carried to an infinitenumber of iterations, will converge to a shifted copy of the log of theoriginal, corresponding to a scaling of the original image after theanti-log is taken. [see D. Brainard and B. Wandell, “An analysis of theRetinex theory of color vision,” Journal Optical Society of America A 3,pp. 1651-1661, October, 1986; and H. K. Rising III, “Analysis andgeneralization of retinex by re-casting the algorithm in wavelets,”Journal of Electronic Imaging 13, pp. 93-99, January, 2004.] In the caseof the McCann'99 algorithm, the reset operation ensures that all imageelements converge to an image in which each pixel is as far from thechosen maximum as the pixel was in the original image from the brightestpoint in the image.

Basic Fractal Image Compression

Fractal image compression is a procedure in which an image is encoded asa set of affine formulas. Each block of the image is matched with thedecimation, both in size and in intensity, of a larger block from theimage, up to the difference in the block averages. The affinetransformation required to map the larger block to the image block isrecorded as part of the compressed file. Together, this set oftransformations forms a mapping of the image into itself. Because eachpart of this transformation is a contraction, the whole transformationis a contraction mapping, in the space of images. [For general referenceto fractal compression, see N. Lu, Fractal Imaging, and M. Barnsley,with Hawley Rising, Fractals Everywhere, 2nd ed.]

The space of images can, and frequently is, endowed with a metric.

This is done every time one calculates rate-distortion curves to measurethe quality of a compression technique. Among the possible choices ofmetrics are the L^(p) metrics, like the Manhattan distance and Euclideandistance, or other more exotic choices like Hausdorff distance, orMahalanobis distance. In the present invention, the Euclidean or L²distance is used, but others, particularly the Hausdorff distance or theL¹ distance, were also used. The decimation method used is the averageof each 2×2 block, but other size blocks could also be used.

The simplest algorithm for fractal compression, then, goes as follows:

1. Decimate a copy of the image, multiplying each new pixel by thecoefficient chosen for contracting the intensity value.

2. For each square block in the original image (4×4 blocks were used)search the possible 4×4 blocks of the decimated copy and find the bestmatch. The criterion for best match will be the smallest square errorbetween the two blocks. (The decimation process takes each 2×2 block ofpixels and averages or chooses from them a single pixel. The 4×4 blocksare used to match blocks of the original to blocks of the decimatedimage to do the fractal transform, after the replacement of the 2×2blocks by individual pixels (decimation) has been done.)

3. For each square block in the original image, save the affine map:

Save the location of the square matched, the location of the squarefound to match it, and the difference in the averages of the two matchedblocks.

The decompression algorithm is much simpler and less time consuming:

1. Starting from a blank screen (it doesn't really need to be blank, onecan start with some other picture if one likes), for each block in theimage, retrieve the block in the same image, and shrink it bothspatially and in intensity in each of the color planes, add thedifference in average, and replace the current block.

2. Iterate the above step sufficiently long for the image to converge.

Because not all the matches are going to be exact, this is a lossycompression algorithm. It converges to the image as it is mapped in thecompression stage, that is, the image as it would look if all thematches found replaced the original pixels. Working with 4×4 blocks, animage of reasonable size takes quite a long time to compress. Running 10to 20 iterations on the decompression should suffice, and this is donein the blink of an eye. There are known methods for making thecompression faster, as well as making the decompression faster, andthere are various other types of improvements to the process that canalso be used [e.g., see N. Lu, Fractal Imaging, Academic Press, MA,1997, which is herein incorporated by reference].

Combining the Two Algorithms

It should be noted that were one to start with a blank screen, andmodify the iterations of the fractal decompression algorithm to averagewith the target block, instead of replacing it, the algorithm must stillconverge. Iterating enough times, the succession of averages convergesto the replacement. Consequently, the algorithm that one would normallyuse for the McCann'99 Retinex, that of averaging with the neighbor of apixel, is replaced by averaging with a pixel from a “similar part” ofthe image. As in the McCann'99, the process will start with the screenat maximum intensity, and as in McCann'99, the process will workentirely on the logarithm of the image, rather than the image valuesthemselves.

There is one more part to include: The nonlinear reset. This can beeffected by resetting any pixels that surpass the maximum duringmapping. The diffusion that is now obtained is potentially verydiscontinuous. The invention is diffusing information about the coloringof a pixel among pixels that resemble a contraction of its neighborhood.As in the McCann'99 algorithm, the invention is again setting pixels bytheir distance from the brightest point in the image, but this time theimage will converge inexorably, with enough iterations, to the original,as it has been programmed to do by the compression algorithm. Theinvention can adjust the rate of convergence spatially compared to therate of convergence in intensity (in each of the 3 color planes, or ofthe 3 dimensional color vector) only by modifying the compression amountin the intensity direction that is used to compress and decompress theimage.

Finally, the algorithm used is as follows (the compression anddecompression algorithms are performed on the logarithm of the image,taking the anti-log at the end of the decompression to render the imageagain):

Compression

1. Take the logarithm of the whole image. (The image or its three colorplanes are replaced by the logarithm of the image, by taking thelogarithm of the intensity either separately, or as the logarithm ofeach component of the color vector.)

2. Create the screen for matching: Decimate the logarithm of the imageby a factor of two in each direction, and apply the contraction value inthe intensity direction. (The intensity here again is in each of thelogarithms of the three color planes, or in the vector intensity of thelogarithm of the 3-dimensional color vector.)

3. Match all the blocks in the original image to this screen, and recordthe mapping.

4. Save this in a file so it can be recovered. This is important sincesophisticated algorithms are not being used for creating this record, soit will take a long time. It is better to have the file around toutilize while examining the decompression algorithm.

The combined retinex/fractal compression algorithm of the invention isillustrated in the flowchart of FIG. 1A. An image is obtained, step 10,either by taking the image with an imaging device, e.g. camera, orreceiving a transmitted image or retrieving it from storage. Thelogarithm of the whole image is taken, step 12. The log of the image isdecimated to create the mapping screen, step 14. The decimation step iscontrolled by the selection of the spatial contraction factor (q,typically 2), step 20, and the intensity contraction factor (p), step22. All the blocks in the original image are matched to the mappingscreen, step 16, to produce the compressed image. The compressed image(mapping) is recorded, step 18, e.g. stored in a file in a computermemory.

Decompression

1. Set the decoding screen to the maximum value.

2. Decompress each block by averaging the contracted block with what isthere.

3. If necessary, reset any pixels that have passed the maximum value.

4. Iterate 2 and 3 until the image converges spatially and in color.

5. Take the anti-log of the result to display the image.

The combined retinex/fractal decompression algorithm of the invention isillustrated in the flowchart of FIG. 1B. The decoding screen is set tothe maximum value, step 30. Each contracted block of the compressedimage is averaged with the current value on the screen, step 32, todecompress each block. Any pixels that have passed the maximum value arereset, step 34. The spatial and color convergence of the image isdetermined, step 36. If the image has not converged spatially and incolor, the algorithm returns to step 32. The loop from step 36 to step32 is repeated until the image has converged spatially and in color. Theanti-logarithm is then taken, step 38, to produce the decompressedimage, which is displayed, step 40, or otherwise output or stored.

Apparatus

FIGS. 2A-B illustrate an apparatus to carry out the invention. System 50of FIG. 2A utilizes an image taken with an imager 51, e.g. a camera. Theimage may have been previously taken and received by transmission orretrieved from storage instead of taken directly from imager 51. Theimage is input into a processor, e.g. digital computer, 52. Memory 53 isassociated with processor 52, and includes program instructions 54 anddata 55. Program instructions 54 are the instructions, which whenexecuted by processor 52, cause the processor to carry out thealgorithms of the invention. Data 55 may include data used in theprocesses, and may include a previously stored compressed image producedby processor 52 from a previously input image that was compressed, andwill be decompressed later. Data 55 may also include a compressed imagethat was obtained from another source and will be decompressed later.Processor 52 uses the program instructions 54 and data 55 in associatedmemory 53 to produce a decompressed image that is output to displaydevice 56 or to another output device, e.g. printer, or storage device.

System 60 of FIG. 2B utilizes an image 61, which is input to an encoder62. Encoder 62 is programmed to carry out the compression of the image,by the combined retinex/fractal compression algorithm. The compressedimage from encoder 62 is either transmitted or stored, as represented bythe transmission/storage element 64. The compressed image is input intodecoder 66. Decoder 66 is programmed to carry out the decompression ofthe compressed image, by the combined retinex/fractal decompressionalgorithm. The decompressed image from decoder 66 is then output tooutput device 68. Encoder 62 and decoder 66 are processors similar toprocessor 52, and may be combined into a single machine 52.

Results

Note that, for simplicity of reproduction, all image figures shownherein are in monochrome, whereas the original images processing wasperformed in color. Using parameters that are “normal” for fractalcompression, in the algorithm above, looks promising on the firstiteration, see FIGS. 3A-B. FIG. 3A is the original image, which is asixteen bit per pixel per plane image. FIG. 3B is the result afterfractal encoding with intensity contraction of ¾ and a singledecompression iteration. (A contraction of ¾ means multiply theintensity by ¾. The range of intensities, were it 0-255 before themultiplication, would now be from 0-191, so it has contracted.) Thecolor has been corrected, but spatially the details are blurred,indicating that the spatial algorithm has not converged. Continuing toiterate immediately exposes a problem with this setting, see FIGS. 4A-B.FIG. 4A is the original image. FIG. 4B is the decompression after 4iterations. Note that by the time the algorithm converges spatially, theimage will be far too dark. The intensity contraction needs to be lessin order to have the image converge spatially before it converges inintensity. Since the function encapsulated by the fractal codesconverges back to the original image, iterating more than is needed toget the proper color adjustment merely returns the image to its originalstate. FIGS. 5A-B also show a sixteen bit image that has been compressedand decompressed using an intensity contraction of ¾. More pronouncedthan the example of FIGS. 3A-B, for this image a single iterationalready has hidden many details. FIGS. 6A-B are based on a change of theintensity contraction to 15/16. The first iteration, FIG. 6A, looks morepromising, but still, by 3 iterations, FIG. 6B, there is a darkening ofthe image in advance of spatial convergence. In this case, only some ofthe detail that actually exists in the image has been captured by thefractal compression.

One experiment was done to test the limits of this variation of theintensity contraction. If the intensity contraction were not there(contraction=1.0) one would expect that, since the conditions forconvergence for the fractal decompression algorithm require acontraction in image space, the image would not decompress. However,certainly the Retinex algorithm converges, and it is buried in theintensity algorithm for the fractal/Retinex scheme. This presents a setof image diffusions: If the image is divided up into orbits, there willbe two kinds:cycles, when the matched block history crosses itself againafter some number of iterations, and “infinite period” cycles, thechains of blocks that do not repeat during the iteration period todecode the image (for infinite iterations, Poincare's recurrence theoremsays that all but a set of measure zero of the points in the image willcycle). On each of these sets, taken as a subimage, the diffusionalgorithm converges, as in the normal Retinex case, to a smooth image.Because of the reset function, and the repeated application of thedifferences when mapping blocks, this will converge to a scaling of theimage, however slowly. The expression for a pixel in the target blockbecomes

$d^{new} = \frac{\left( {d^{old} + \left( {r^{c} + \left( {r - d} \right)} \right)} \right)}{2}$where r^(c) is the contracted substitution block, and r−d refers to theoriginal stored difference in means. This formula is identical to theformula for pixel replacement in the McCann'99 algorithm, except thatinstead of an average with a manipulation of a neighboring block, theaveraging is done with a similar block possibly a discontinuous jumpaway from the target.

This form of the algorithm was tested. It was found that the image doesindeed converge, albeit very slowly, and the spatial convergence is muchfaster than the intensity convergence, see FIGS. 7A-B and FIGS. 8A-B.FIG. 7A is the original image of the Stanford quadrangle. Using acontraction of 1 (i.e., no contraction) during the fractal decompressionyields a spatially converged image with the intensities just beginningto move at 50 iterations, FIG. 7B. FIG. 8A is an image of the Stanfordquadrangle, with a contraction of 1 in intensity, at 1000 iterations.Note—for simplicity of reproduction monochrome images are shown in thedrawings. The colors are beginning to converge and the spatialinformation, which contracts quickly here, is already sharp. At 2000iterations, depicted in FIG. 8B, the colors were nearly converged, withthe detail in the foreground visible, and the colors of some featuresnearly correct. Without the contraction in the intensity directions, thespatial convergence is significantly faster than the intensityconvergence. Will it converge to the recolored image properly? It shouldbe noted that after 2000 iterations, the color of the sky was justbeginning to turn blue, and the color of the flowers on the far side ofthe quadrangle, were seen to be clearly red. Thus, the correct colorswere present, although the foreground and sky had not yet converged inthese images, and the detail that is available, after the spatialconvergence is nearly complete, was excellent.

Clearly, this convergence is too slow to be practical, and is done fortwo reasons. For the earlier images, one can see that intensityconvergence of ¾ is too quick, as is 15/16. For a convergence of 1, itis too slow, so there is a factor that is between these values, that isthe correct rate, at least for this image and this change in dynamicrange.

The algorithm of the invention, which combines the McCann'99 Retinexalgorithm with the operation of fractal compression and decompression,is interesting for several features. It does the recoloring of theimage, without an assumption of piecewise smoothness, which is inherentin most color constancy algorithms. In fact, the iteration formula isdiscontinuous, and pixels mix only within their block orbits.Consequently, this allows a fair degree of randomness in the way thatcells, for instance in the retinal ganglia, or even in the cortex mustbe connected. Since the algorithm essentially transmits edge informationacross the resolutions of the edges detected, the process could occurjust as easily in a system in the cortex, for instance in V1 (Visualarea 1 of the brain, also known as Area 17, the visual cortex, or thestriate cortex) where edge detectors exist at many different resolutions[S. Zeki, A Vision of the Brain, Blackwell Scientific Publications, MA,1993]. This is not to conjecture that this is where it happens, but onlyto make the argument that without the necessity of continuity, there aremany places in which color correction can occur. On the other hand, itis obvious from the presentation of the algorithm itself, and its lowcontinuity, that the simultaneous contrast effects of the Retinexalgorithm cannot be expected. FIG. 7A-B and FIG. 8A-B, do show an earlycoloring of strong edges; however, this effect is much less than withstandard (and more continuous) Retinex algorithms.

Additional features and alternate embodiments may also be implementedwith the invention. The Frankle-McCann algorithm could be used withfractal decompression. A trick to decoding called pixel chaining [N. Lu,Fractal Imaging, Academic Press, MA, 1997], which calculates individualpixels along block orbits, could be used in the process. Narrowing thebracket of contractions for the intensity dimensions may also be done.

Although the description above contains many details, these should notbe construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodimentsof this invention. Therefore, it will be appreciated that the scope ofthe present invention fully encompasses other embodiments which maybecome obvious to those skilled in the art, and that the scope of thepresent invention is accordingly to be limited by nothing other than theappended claims, in which reference to an element in the singular is notintended to mean “one and only one” unless explicitly so stated, butrather “one or more.” All structural, chemical, and functionalequivalents to the elements of the above-described preferred embodimentthat are known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe present claims. Moreover, it is not necessary for a device or methodto address each and every problem sought to be solved by the presentinvention, for it to be encompassed by the present claims. Furthermore,no element, component, or method step in the present disclosure isintended to be dedicated to the public regardless of whether theelement, component, or method step is explicitly recited in the claims.No claim element herein is to be construed under the provisions of 35U.S.C. 112, sixth paragraph, unless the element is expressly recitedusing the phrase “means for.”

1. An image processing method, comprising: obtaining an image within animage processing system configured for executing program instructionsfrom a computer readable medium or memory executed by a processor; andprocessing the image, within the image processing system, by acombination of retinex processing and fractal compression anddecompression techniques.
 2. The method of claim 1, wherein processingthe image comprises: encoding the image to produce a compressed image;and decoding the compressed image to produce a decompressed image. 3.The method of claim 1, further comprising outputting the processedimage.
 4. The method of claim 1: wherein dynamic range of thedecompressed image is controlled by the retinex processing; and whereinsize of the decompressed image is controlled by the fractal compressionand decompression.
 5. The method of claim 4, further comprising:outputting the decompressed image to an output device; wherein dynamicrange and size of the decompressed image are controlled to match dynamicrange and resolution of the output device.
 6. The method of claim 1,wherein processing the image comprises: replacing the image with afunction which contracts the image by a spatial contraction factor q ineach spatial dimension and by a color compression factor p in each colordimension, by dividing the image into image blocks, averaging each imageblock with a target, and selecting p to be sufficiently close to one, sothat spatial components of the function converge faster than colorcomponents so that the converged image has independently controlled sizeand color.
 7. The method of claim 1, wherein processing the imagecomprises: compressing the image by a compression algorithm whichcombines retinex processing with fractal compression to produce acompressed image; decompressing the compressed image by a decompressionalgorithm which combines retinex processing with fractal decompressionto produce a decompressed image of a desired dynamic range and size; andoutputting the decompressed image.
 8. The method of claim 7, furthercomprising compressing the image by a conventional fractal compressionalgorithm which is modified by decreasing the compression factor in thecolor dimensions.
 9. The method of claim 7, further comprisingdecompressing the image by a conventional fractal decompressionalgorithm which has been modified so that range blocks are averaged downand then combined with domain blocks by averaging in, using the rangeblock values and difference as one part of the average, and the existingdomain blocks as the other.
 10. The method of claim 7, furthercomprising outputting the decompressed image to a display device orprinter having a dynamic range and resolution that is matched by thedynamic range and size of the decompressed image.
 11. The method ofclaim 7, further comprising compressing the image by: taking thelogarithm of the whole image; creating a mapping screen by decimatingthe logarithm of the image by a factor of q in each spatial dimensionand applying a contraction factor p to the intensity in each colordimension; matching all blocks in the image to the mapping screen; andrecording the mapping.
 12. The method of claim 11, further comprisingdecompressing the compressed image by: setting a decoding screen to amaximum value; decompressing each block of the compressed image byaveraging the compressed block with the current value on the decodingscreen; resetting any pixels that exceed a maximum value; iterating thedecompressing and resetting steps until the image has convergedspatially; and taking the anti-logarithm of the converged image.
 13. Themethod of claim 11, further comprising: controlling the dynamic range ofthe decompressed image by the factor p; and controlling the size of thedecompressed image by the factor q.
 14. The method of claim 13, furthercomprising: outputting the decompressed image to an output device;wherein the dynamic range and size of the decompressed image arecontrolled to match the dynamic range and resolution of the outputdevice.
 15. An apparatus for processing an image, comprising: aprocessor having an input for receiving an image; a memory associatedwith the processor, the memory including: program instructions, whichwhen executed by the processor, cause the processor to process the imageby a combination of retinex processing and fractal compression anddecompression techniques; and data used to process the image.
 16. Theapparatus of claim 15, wherein the processor comprises a digitalcomputer.
 17. The apparatus of claim 15, further comprising an outputdevice connected to the processor.
 18. The apparatus of claim 15,wherein the program instructions comprise: instructions for compressingthe image by a compression algorithm which combines retinex processingwith fractal compression to produce a compressed image; and instructionsfor decompressing the compressed image by a decompression algorithmwhich combines retinex processing with fractal decompression to producea decompressed image of a desired dynamic range and size.
 19. Theapparatus of claim 18, further comprising: an output device connected tothe processor to receive the decompressed image; wherein the dynamicrange and size of the decompressed image are controlled to match thedynamic range and resolution of the output device.
 20. An apparatus forprocessing an image, comprising: an encoder having an input forreceiving the image and programmed to compress the image by a combinedretinex and fractal compression algorithm; and a decoder having an inputfor receiving the compressed image and programmed to decompress thecompressed image by a combined retinex and fractal decompressionalgorithm.
 21. The apparatus of claim 20, further comprising atransmission element connected between the encoder and decoder totransmit the compressed image from the encoder to the decoder.
 22. Theapparatus of claim 20, further comprising a storage element connectedbetween the encoder and decoder to store the compressed image from theencoder until sent to the decoder.
 23. The apparatus of claim 20,further comprising an output device connected to the decoder to receivethe decompressed image from the decoder.
 24. An image processor,comprising: means for obtaining an image; means for processing the imageby a combination of retinex processing and fractal compression anddecompression techniques; and means for outputting the processed image.25. The image processor of claim 24, wherein the means for processingthe image comprises: means for encoding the image to produce acompressed image; and means for decoding the compressed image to producea decompressed image.
 26. The image processor of claim 24, wherein themeans for processing the image comprises: means for compressing theimage by a compression algorithm which combines retinex processing withfractal compression to produce a compressed image; and means fordecompressing the compressed image by a decompression algorithm whichcombines retinex processing with fractal decompression to produce adecompressed image of a desired dynamic range and size.
 27. A machinereadable medium containing instructions, which when executed by amachine, cause the machine to perform operations comprising: processingan image, within an image processing system configured for executingprogram instructions executed by a processor, by a combination ofretinex processing and fractal compression and decompression techniques;and outputting the processed image.
 28. The medium of claim 27, whereinprocessing the image comprises: encoding the image to produce acompressed image; and decoding the compressed image to produce adecompressed image.
 29. The medium of claim 27, wherein processing theimage comprises: compressing the image by a compression algorithm whichcombines retinex processing with fractal compression to produce acompressed image; and decompressing the compressed image by adecompression algorithm which combines retinex processing with fractaldecompression to produce a decompressed image of a desired dynamic rangeand size.
 30. The medium of claim 29, wherein the image is compressedby: taking the logarithm of the whole image; creating a mapping screenby decimating the logarithm of the image by a factor of q in eachspatial dimension and applying a contraction factor p to the intensityin each color dimension; matching all blocks in the image to the mappingscreen; and recording the mapping.
 31. The medium of claim 30, whereinthe compressed image is decompressed by: setting a decoding screen to amaximum value; decompressing each block of the compressed image byaveraging the compressed block with the current value on the decodingscreen; resetting any pixels that exceed a maximum value; iterating thedecompressing and resetting steps until the image has convergedspatially; and taking the anti-logarithm of the converged image.